Fresnel assembly for light redirection in eye tracking systems

ABSTRACT

A head-mounted device (HMD) comprises a display element, a Fresnel assembly, an illumination source, and a camera assembly. The display element outputs image light in a first band of light through a display surface. The optics block directs light from the display element to a target region (e.g., includes a portion of a user&#39;s face, eyes, etc.). The Fresnel assembly transmits light in the first band and directs light in a second band different than the first band to a first position. The source illuminates the target area with light in the second band. The camera is located in the first position and captures light in the second band corresponding to light reflected from the target area that is then reflected by the Fresnel assembly toward the camera. A controller may use the captured light to determine tracking information for areas of a user&#39;s face (e.g., eyes).

BACKGROUND

The present disclosure relates generally to light redirection, andspecifically relates to a Fresnel assembly for light redirection in eyetracking systems.

Eye tracking refers to the process of detecting the direction of auser's gaze, which may comprise detecting an orientation of an eye in3-dimentional (3D) space. Eye tracking in the context of headsets usedin, e.g., virtual reality and/or augmented reality applications can bean important feature. Conventional systems commonly use, e.g., a numberof infrared light sources to illuminate the eye light, and a camera isused to image a reflection of the light sources from the eye.Traditionally, eye tracking systems use beam splitters to redirectinfrared light reflected from the eye to the camera. However, beamsplitters are often large, cumbersome, and unsuitable for HMDs used inaugmented reality (AR), mixed reality (MR), and virtual reality (VR)systems.

SUMMARY

A Fresnel assembly transmits (e.g., partially or fully) light in a firstband (e.g., visible light) and redirects some, or all of light in asecond band (e.g., infrared light) to one or more locations. The Fresnelassembly includes a plurality of surfaces that act to redirect light inthe second band. The plurality of surfaces may be coated with a dichroicmaterial that is transmissive in the first band and reflective in thesecond band. In some embodiments, an immersion layer is overmolded ontothe Fresnel assembly to form an immersed Fresnel assembly. The immersionlayer may be index matched to the Fresnel assembly. Additionally, insome embodiments, one or more surfaces of the immersed Fresnel assemblymay be shaped (e.g., concave, convex, asphere, freeform, etc.) to adjustoptical power of the immersed Fresnel assembly. The Fresnel assembly maybe integrated into a head-mounted display (HMD).

In some embodiments, a HMD includes a display element, an optics block,a Fresnel assembly, an illumination source, and a camera assembly. Thedisplay element outputs image light in a first band (e.g., visiblelight) of light through a display surface of the display element. Theoptics block directs light from the display element to an eyebox (aregion in space occupied by an eye of the user). The Fresnel assemblytransmits light in the first band and directs light in a second band(e.g., infrared light) different than the first band to a firstposition. The illumination source (e.g., part of a tracking system)illuminates a target region (eyes and/or portion of the face) with lightin the second band. The camera (e.g., part of an eye and/or facetracking system) is located in the first position, and is configured tocapture light in the second band corresponding to light reflected fromthe target region of the user and reflected by the Fresnel assembly.

In some embodiments, the Fresnel assembly may be non-wavelengthsensitive (e.g. it could be a non-immersed, uncoated surface, or itcould have a partially-reflective coating and is immersed). It transmitsa portion of the light from the display and directs it to the eyebox. Italso directs parts of the display light outside of the eyebox. Thecamera is positioned such that the light that comes from the eye isreflected off of the Fresnels and is captured by the camera.Additionally, in some embodiments, the HMD includes a controller (e.g.,part of the tracking system) that generates tracking information (e.g.,gaze location and/or facial expressions).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a cross section of a portion of a display blockof an HMD (not shown), in accordance with an embodiment.

FIG. 2 is a diagram of a cross section of a portion of a display blockof an HMD (not shown) with a canted Fresnel assembly, in accordance withan embodiment.

FIG. 3 is a diagram of a cross section of a portion of a display blockof an HMD (not shown) with an immersed Fresnel assembly, in accordancewith an embodiment.

FIG. 4 is a diagram of a cross section of a portion of an immersedFresnel assembly 410, in accordance with an embodiment.

FIG. 5 is a diagram of a cross section of a portion of an immersedFresnel assembly that captures multiple view angles, in accordance withan embodiment.

FIG. 6 is a diagram of a cross section of a portion of an immersedFresnel assembly configured to interact with multiple positions, inaccordance with an embodiment.

FIG. 7A is a diagram of an HMD, in one embodiment.

FIG. 7B is a diagram of a cross-section of the HMD, in one embodiment.

FIG. 8A is an example array of sub-pixels on an electronic displayelement, in accordance with an embodiment

FIG. 8B is an image of an example array of sub-pixels adjusted by anoptical block, in accordance with an embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION

FIG. 1 is a diagram of a cross section 100 of a portion of a displayblock of an HMD (not shown), in accordance with an embodiment. As shownin FIG. 1, the display block includes a display element 110, a Fresnelassembly 120, and a tracking system 130. In some embodiments, thedisplay block 100 may also include an optics block 140.

The display element 110 displays images to the user. In variousembodiments, the display element 110 may comprise a single electronicdisplay panel or multiple electronic display panels (e.g., a display foreach eye of a user). Examples of the display element 110 include: aliquid crystal display (LCD), an organic light emitting diode (OLED)display, an active-matrix organic light-emitting diode display (AMOLED),a quantum organic light emitting diode (QOLED) display, a quantum lightemitting diode (QLED) display, a transparent organic light emittingdiode (TOLED) display, some other display, or some combination thereof.In some embodiments, the display element 110 is a waveguide display. Thedisplay element 110 emits content within a first band of light (e.g., ina visible band).

The Fresnel assembly 120 transmits light in the first band and reflectslight within a second band. For example, the Fresnel assembly 120 maytransmit light within a visible band (e.g., 400-700 nanometers (nm)),and may reflect light within an infrared (IR) band (e.g., above 780 nm).The Fresnel assembly 120 comprises a first surface that includesplurality of reflective surfaces that form a Fresnel lens, a portion 145of the plurality of reflective surfaces are illustrated in FIG. 1. Insome embodiments, the plurality of reflective surfaces are coated with adichroic film that is transmissive in the first band and reflective inthe second band. In other embodiments, the plurality of the reflectivesurfaces are coated with a partial reflective coating. As discussed indetail below with regard to, e.g., FIGS. 4-6 the plurality of reflectivesurfaces can be configured to reflect light in the second band todifferent positions, and the positions. The Fresnel assembly 120 canpartially transmit light, and partially reflect light, for any lightband of interest. For example, without coating the Fresnel assembly 120can weakly reflect light by Fresnel reflection, and transmit the rest ofthe light. In contrast, with coatings, the transmission/reflection ratiocan be adjusted. Alternatively, the Fresnel assembly 120 can transmitlight with one polarization, and reflect light with an orthogonalpolarization. This can be useful especially for polarization basedviewing optics, where light is transmitted in one single polarization.Additionally, while not shown in FIG. 1, in some embodiments, theFresnel assembly 120 is coupled directly to a display surface of to thedisplay element.

The Fresnel assembly 120 is at least partially transmissive to light inthe first band. In some embodiments, the Fresnel assembly 120 mayincrease or decrease the optical power of light in the first band fromthe display element 110. Additionally, for light in the first band, theFresnel assembly 120 may reduce fixed pattern noise (i.e., the screendoor effect). As is described in the descriptions of FIGS. 8A and 8B,the Fresnel assembly 120 blurs light emitted from individual pixels inthe display element 110. This blurring allows for dark spots betweenindividual colors to be masked.

The optics block 140 directs light to a target region. The target regionincludes a portion of the user's face. In some embodiments, the targetregion includes one or both eyes (e.g., the eye 155) of the user. Forease of illustration, the target region in FIG. 1 is represented as aneyebox 150. The eyebox 150 is a region in space that is occupied by aneye 155 of a user of the HMD. In an embodiment, the optics block 140includes one or more optical elements and/or combinations of differentoptical elements. For example, an optical element is an aperture, aFresnel lens, a convex lens, a concave lens, a filter, or any othersuitable optical element that affects the image light emitted from thedisplay element 110. In some embodiments, one or more of the opticalelements in the optics block 140 may have one or more coatings, such asanti-reflective coatings.

Magnification of the image light by the optics block 140 allows thedisplay element 110 to be physically smaller, weigh less, and consumeless power than larger displays. Additionally, magnification mayincrease a field of view of the displayed content. For example, thefield of view of the displayed content is such that the displayedcontent is presented using almost all (e.g., 110 degrees diagonal), andin some cases all, of the user's field of view. In some embodiments, theoptics block 140 is designed so its effective focal length is largerthan the spacing to the display element 110, which magnifies the imagelight projected by the display element 110. Additionally, in someembodiments, the amount of magnification is adjusted by adding orremoving optical elements.

In some embodiments, the optics block 140 is designed to correct one ormore types of optical errors. Examples of optical errors include:two-dimensional optical errors, three-dimensional optical errors, orsome combination thereof. Two-dimensional errors are optical aberrationsthat occur in two dimensions. Example types of two-dimensional errorsinclude: barrel distortion, pincushion distortion, longitudinalchromatic aberration, transverse chromatic aberration, or any other typeof two-dimensional optical error. Three-dimensional errors are opticalerrors that occur in three dimensions. Example types ofthree-dimensional errors include spherical aberration, comaticaberration, field curvature, astigmatism, or any other type ofthree-dimensional optical error. In some embodiments, content providedto the display element 110 for display is pre-distorted, and the opticsblock 140 corrects the distortion when it receives image light from theelectronic display element 110 generated based on the content.

The tracking system 130 tracks movement of the target region. Some orall of the tracking system 130 may or may not be in a line of sight of auser wearing the HMD. The tracking system 130 is typically locatedoff-axis to avoid obstructing the user's view of the display element110, although the tracking system 130 may alternately be placedelsewhere. Also, in some embodiments, there is at least one trackingsystem 130 for different portions of the user's face (e.g., a trackingsystem for the user's left eye and a second tracking system for theuser's right eye). In some embodiments, only one tracking system 130 maytrack multiple target regions.

The tracking system 130 may include one or more illumination sources160, a camera assembly 165, and a controller 170. The tracking system130 determines tracking information using data (e.g., images) capturedby the camera assembly 165 of the target region (e.g., the eye 155).Tracking information describes a position of a portion of the user'sface. Tracking information may include, e.g., facial expressions, gazeangle, eye orientation, inter-pupillary distance, vergence depth, someother metric associated with tracking an eye, some other metricassociated with tracking a portion of the user's face, or somecombination thereof. Some embodiments of the tracking unit havedifferent components than those described in FIG. 1.

The illumination source 160 illuminates the target region (e.g., theeyebox 150) with light in the second band of light (i.e., source light175) that is different from the first band of light associated withcontent from the display element 110. Examples of the illuminationsource 160 may include: a laser (e.g., a tunable laser, a continuouswave laser, a pulse laser, other suitable laser emitting infraredlight), a light emitted diode (LED), a fiber light source, a lightguide, another other suitable light source emitting light in the secondband, or some combination thereof. In some embodiments, the illuminationsource can be inside the field of view of the display, and istransparent to the display light. For example, the illumination sourcecan be a transparent light guide with small light extraction features ontop. In various embodiments, the illumination source 160 may also beconfigured to emit light in the first band. In some embodiments, thetracking system 130 may include multiple illumination sources 160 forilluminating one or more portions of the target region. In someembodiments, the light emitted from the one or more illumination sources160 is a structured light pattern.

Reflected light 180 (inclusive of scattered light) from the illuminatedportion of the target region (e.g., the eye 155) is light in the secondband that is reflected by the target region towards the Fresnel assembly120. The reflected light 162 is redirected by the Fresnel assembly 120toward a first position, and this redirected light is referred to asredirected light 185. Some or all of the redirected light 185 iscaptured by the camera assembly 165 that is located at the firstposition.

The camera assembly 165 captures images of the target region (e.g., theeye 155 in the eyebox 150). The camera assembly 165 includes one or morecameras that are sensitive to light in at least the second band (i.e.,light emitted by the illumination source 160). In some embodiments, theone or more cameras may also be sensitive in other bands of light (e.g.,the first band). In some embodiments, a camera may be based onsingle-point detection (e.g., photodiode, balanced/matched photodiodes,or avalanche photodiode), or based on one or two-dimensional detectorarrays (e.g., linear photodiode array, CCD array, or CMOS array). Insome embodiments, the sensor plane of the camera is tilted with regardsto the camera's lens, following the Scheimpflug condition, such that theimage can be in focus across the whole sensor plane. In someembodiments, the camera assembly 165 may include multiple cameras tocapture light reflected from the target region. The camera assembly 165includes at least one camera located at the first position. Accordingly,the camera assembly 165 captures images of the target region (e.g., theeye 155) using the redirected light 164.

The controller 170 determines tracking information using images from thecamera assembly 165. For example, in some embodiments, the controller170 identifies locations of reflections of light from the one or moreillumination sources 160 in an image of the target region. Thecontroller 170 determines a position and an orientation of the eye 155and/or portions of the face in the target region based on the shapeand/or locations of the identified reflections. In cases where thetarget region is illuminated with a structured light pattern, thecontroller 170 can detect distortions of the structured light patternprojected onto the target region, and can estimate a position and anorientation of the eye 155 and/or portions of the face based on thedetected distortions. The controller 170 can also estimate a pupillaryaxis, a gaze angle (e.g., corresponds to a foveal axis), a translationof the eye, a torsion of the eye, and a current shape of the eye 155based on the image of the illumination pattern captured by the camera155.

Note that in the illustrated embodiment, the reflected light 180 passesthrough the optics block 140 prior to incidence on the Fresnel assembly120. In alternate embodiments (not shown), the Fresnel assembly 120 iscloser to the eyebox 150 than the optics block 140 (i.e., a distancebetween the optics block 140 and the display element 110 is less than adistance between the Fresnel assembly 120 and the display element 110)and light in the second band reflected from the eye 155 does not passthrough an optics block 140 before it is directed by the Fresnelassembly 120 to a first position.

FIG. 2 is a diagram of a cross section 200 of a portion of a displayblock of an HMD (not shown) with a canted Fresnel assembly 210, inaccordance with an embodiment. The portion of the display block includesthe display element 110, the canted Fresnel assembly 210, the opticsblock 140, and the tracking system 130. The canted Fresnel assembly 210is substantially the same as the Fresnel assembly 120, except that it iscanted with respect to an optical axis 220 and is positioned at a firstdistance 230 from the display element 110.

The optical axis 220 passes through the display element 110, the cantedFresnel assembly 210, and the optics block 140. In some embodiments, theoptical axis 220 bisects one or more of the display element 110, thecanted Fresnel assembly 210, the optics block 140, or some combinationthereof. The canted Fresnel assembly 210 is positioned at a tilt angle,α, with respect to the optical axis 220. The tilt angle is determined bysystem level trade-offs between stray light from the draft facets, hownormal the camera's view of the eye is needed, how compact the Fresnelassembly needs to be, etc. Reasonable tilt angles would be smaller than30 degrees.

In this embodiments, the canted Fresnel assembly 210 is substantiallycloser to the optics block 140 than the display element 110. The cantedFresnel assembly 210 is located the first distance 230 from the displayelement 110, and a second distance 240 from the optics block 140. Thefirst distance 230 is substantially larger than the second distance 240.If the Fresnel assembly is canted, it's preferred to put it behind theoptics block 140, and with a large distance from the display. If it's inbetween the eye and the optics block 140, it wants to be close tonon-canted such that it would not increase the design eye-relief. Whenit's put behind the optics block 140, it is usually better to putFresnel surfaces far away from the display, such that the eye would notbe able to focus on the Fresnel grooves or imperfections of immersion.If the Fresnels/prisms also function as screen-door-reduction films,then it can potentially be put very close to the display.

FIG. 3 is a diagram of a cross section 300 of a portion of a displayblock of an HMD (not shown) with an immersed Fresnel assembly 310, inaccordance with an embodiment. The portion of the display block includesthe display element 110, the immersed Fresnel assembly 310, and thetracking system 130.

The immersed Fresnel assembly 310 is the Fresnel assembly 120 overmoldedwith an immersion layer 320. The immersed Fresnel assembly 310 iscoupled to the display element 110. In some embodiments, the immersedFresnel assembly 310 is directly coupled to a display surface of thedisplay element 110. In alternate embodiments, the immersed Fresnelassembly 310 and the display element 110 are separated by one or moreintermediate layers (e.g., optical films). The immersed Fresnel assembly310 can be located in any location between the display 110 and the eye155.

The immersion layer 320 is index matched to a material that makes up theFresnel assembly 120. As the immersion layer 320 is index matched to theFresnel assembly 120, light in the first band from the display element110 is not affected by the Fresnel assembly 120. For example, due to thesubstantially matched index, light in the first band is not refracted bythe plurality of surfaces (e.g., surface 330) at the interface betweenthe immersion layer 320 and the Fresnel assembly 120. Instead light inthe first band interacts with the immersed Fresnel assembly 310 as ablock of a material of a single index. The immersion layer can be a glueor UV curable material that is of similar refractive index to theFresnels. Or, a different process can be, two matching Fresnels can besandwiched together, with index-matching glue in between. In someembodiments, both match Fresnels are the same material (such as a UVcurable material), and no index-match glue is needed in between.

The immersed Fresnel assembly 310 includes an outer surface 340 (e.g.,substrate). In this embodiment, the outer surface 340 is flat. Inalternate embodiments, the outer surface 340 is shaped to provide someadjustment to optical power of light being transmitted by the immersedFresnel assembly 310. For example, the output surface may be concave,convex, an asphere, spherical, a freeform surface, flat, or some othershape. In some embodiments, the Fresnel assembly 120 is coupled to on asubstrate that is not flat (e.g., curved, or some other shape).

In this embodiment, the plurality of surfaces of the Fresnel assembly120 have a dichroic coating that is transmissive in the first band oflight (e.g., visible light), but is reflective in the second band oflight (e.g., IR light). In some embodiments, the coatings may besputtered onto one or more of the surfaces, laminated onto one or moreof the surfaces, etc. Accordingly, as discussed below with regard toFIGS. 4-6, light in the second band incident on the surfaces arereflected (e.g., toward the camera assembly 155). In some embodiments,the coating maybe a partial reflective coating. In other embodiments,the coating may be a polarization beam splitter coating. Moreover, insome embodiments, one or more of junctions between surfaces of theFresnel assembly 120 may be softened (i.e., smoothed to have a radius ofcurvature v. a sharp edge formed by two intersecting surfaces).Softening of a junction can facilitate lamination of a coating onto theplurality of surfaces of the Fresnel structure.

FIG. 4 is a diagram of a cross section 400 of a portion of an immersedFresnel assembly 410, in accordance with an embodiment. The portion ofthe immersed Fresnel assembly 410 includes a portion of the Fresnelassembly 120 and a portion of the immersion layer 320 that is coupled tothe Fresnel assembly 120.

Light 430 in the second band is incident on a surface 420 of theplurality of surfaces of the Fresnel assembly 120. In this embodiment,the plurality of surfaces have a dichroic coating that is transmissivein the first band and reflective in the second band. Accordingly, thelight 430 is reflected toward a first position (e.g., occupied a cameraof the camera assembly 165). Some or all of the surfaces of the Fresnelassembly 120 may be optimized to reduce stray light. For example, aslope of a draft facet 440 may be shaped such that it aligns with achief ray angle exitance ray going into the camera's entrance pupil.Note that light might hit the wrong Fresnel facet in two scenarios, onits way to the Fresnel, and after it gets reflected by the Fresnel. Insome embodiments where the view of the eye 155 is close to normal, andrays hit the Fresnel assembly 120 at close to normal and rarely hit thewrong Fresnel facet on the way in, so it's better to align the chief raysuch that it would avoid hitting the wrong Fresnel facet on the way outand into the camera assembly 165. In other embodiments, some tradeoffsmay be made such that stray light can be minimized considering bothscenarios. An alternative way to remove stray light is to deactivate the“unused” Fresnel draft facet, such that it wouldn't be reflective andgive stray light. In some embodiments, only the “useful” facet is coatedto reflect light, and the draft facet is not coated (this can be done bydirectional coating techniques) or painted black. Note that prior toarriving at a camera of the camera assembly 165, in the immersed Fresnelassembly 410, the light 430 is refracted at the outer surface 340.Accordingly, by using a high index material (e.g., 2.5) for theimmersion layer 320 and the Fresnel assembly 120, the light 430 can bebent at a greater angle relative to a low index material (e.g., 1.3).And a greater bending of the light 430 may be useful in reducing a formfactor of the HMD.

Another artifact to avoid is Fresnel visibility to the view's eye. Ifthe “unused” draft facet reflects a lot of visible light into theviewer's eye, the Fresnel facets may be visible. In some embodiments,optimum co-designs of the coating and the Fresnel draft angle mitigatesthis artifact. In alternative embodiments, the draft facet is uncoatedand becomes invisible for the see-through transmission path. In otherembodiments, the Fresnel pitch is small (smaller than 400 microns) suchthat it is difficult for the eye to resolve the individual facets.

Note that in alternate embodiments, the first position is also occupiedby the illumination source 165. In these embodiments, the illuminationsource 160 illuminates the Fresnel assembly 120 with light in the secondband, and the Fresnel assembly 120 redirects the light toward an eyebox.For further suppression of stray light into the camera, in someembodiments, crossed polarizers or other polarization diversity methodsare used to block unwanted illumination light hitting the camera.

FIG. 5 is a diagram of a cross section 500 of a portion of an immersedFresnel assembly 505 that captures multiple view angles, in accordancewith an embodiment. The portion of the immersed Fresnel assembly 505includes a portion of a Fresnel assembly 510 and a portion of theimmersion layer 320 that is coupled to the Fresnel assembly 510. TheFresnel assembly 510 is substantially similar to the Fresnel assembly120, except that, e.g., the plurality of surfaces are configured todirect light in a second band from different angles to the cameraassembly 165. In this manner the camera assembly 165 is able to captureusing a single camera in a single image frame multiple view angles of atarget region (e.g., eye, portion of the face, or both). In someembodiments, the camera assembly 165 may capture three dimensional (3D)information that describes a portion of the eye 155.

In this embodiment, the Fresnel assembly 510 includes a plurality ofsurfaces. The plurality of surfaces have a dichroic coating that istransmissive in the first band and reflective in the second band. Light515 in the second band is incident on the outer surface 340 of theimmersion layer 320 at a first angle. The light 515 is refracted at theouter surface 340 toward a surface 520 of the plurality of surfaces ofthe Fresnel assembly 510. The surface 520 reflects the light toward asurface 525 of the plurality of surfaces, and the surface 525 reflectsthe light 515 toward the outer surface 340 of the immersion layer 320.The light 515 refracts at a refraction point 530 and propagates toward afirst position that is occupied by a camera (e.g., a camera of thecamera assembly 165). Note that the light might not follow the samepaths in alternative embodiments, but in those cases, light coming fromdifferent paths can also allow for multiple views of the eye 155, orallow for paths where an illuminator can be inserted.

Concurrent with the light 515, light 535 in the second band is incidenton the outer surface 340 of the immersion layer 320 at a second anglethat is different than the first angle. The light 535 is refracted atthe outer surface 340 toward a surface 540 of the plurality of surfacesof the Fresnel assembly 510. The surface 540 reflects the light 535 backtoward a portion of the outer surface 340 at an angle such that totalinternal reflection occurs at a TIR point 545. The light 535 isreflected toward a surface 550 of the plurality of surfaces, and thesurface 550 reflects the light 535 toward the outer surface 340 of theimmersion layer 320. The light 535 refracts at a refraction point 555and propagates toward a first position that is occupied by a camera(e.g., a camera of the camera assembly 165).

Note that the light 515 and the light 535 are representative ofdifferent view angles of a target area (e.g., eye and/or portion of theface) being imaged by the camera. And that the camera is able to capturein a single image frame both view angles of the eye. The tracking system130 may use the captured images for generating tracking information.

In alternate embodiments, a first set of surfaces are used for one view,and another set of surface is used for a different view. For example,the surfaces of the first set may be shaped to capture lightspecifically for a particular view, and the surface of the second setmay be shaped to capture light specifically for a different view. Inthis embodiments, the surfaces of the first set and the surfaces of thesecond set may be positioned in separate portions of the Fresnelassembly 510, interlaced across the entire Fresnel assembly 510, etc.Additionally, in some embodiments, the surfaces may be shaped and/orcoated such that different polarizations of light and/or wavelengths oflight correspond to different view angles that are captured by thecamera.

FIG. 6 is a diagram of a cross section 600 of a portion of an immersedFresnel assembly 605 configured to interact with multiple positions, inaccordance with an embodiment. The portion of the immersed Fresnelassembly 605 includes a portion of a Fresnel assembly 610 and a portionof the immersion layer 320 that is coupled to the Fresnel assembly 610.The Fresnel assembly 610 is substantially similar to the Fresnelassembly 120, except that, e.g., the plurality of surfaces include afirst set of surfaces associated with a first position 615 and a secondset of surfaces associated with a second position 620 that is differentfrom the first position 615. The first position 615 is on a first sideof the optical axis 220, and the second position 615 is on a second sideof the optical axis 220. In alternate embodiments, the first position615 and the second position 620 are symmetric with respect to theoptical axis 220.

The first position 615 is occupied by a first device 625 and the secondposition 620 is occupied by a second device 630. The first device 625and the second device 630 may be a camera (e.g., of the camera assembly165), an illumination source 160, a display panel, or some combinationthereof. The illumination source or the display panel can be LEDs,microLEDs, MicroOLEDs, display panels that show an arbitrary pattern(e.g., such as dots or sinusoidal patterns), some other suitableilluminators, or some combination thereof. However, at least one of thefirst position 615 or the second position 620 includes a camera. Asillustrated in FIG. 6, the first device 625 is a camera, and the seconddevice 630 may be a camera or an illumination source 160. In analternate embodiment, both the first position 615 and the secondposition 620 include cameras of the camera assembly 165. In an alternateembodiment, the first position 615 includes a camera of the cameraassembly 165 and the second position 620 includes an illumination source160.

FIG. 7A shows a diagram of the HMD, as a near eye display 700, in oneembodiment. In this embodiment, the HMD is a pair of augmented realityglasses. The HMD 700 presents computer-generated media to a user andaugments views of a physical, real-world environment with thecomputer-generated media. Examples of computer-generated media presentedby the HMD 700 include one or more images, video, audio, or somecombination thereof. In some embodiments, audio is presented via anexternal device (e.g. speakers and headphones) that receives audioinformation from the HMD 700, a console (not shown), or both, andpresents audio data based on audio information. In some embodiments, theHMD 700 may be modified to also operate as a virtual reality (VR) HMD, amixed reality (MR) HMD, or some combination thereof. The HMD 700includes a frame 710 and a display assembly 720. In this embodiment, theframe 710 mounts the near eye display 700 to the user's head. Thedisplay 720 provides image light to the user.

FIG. 7B shows a cross-section view 730 of the near eye display 700. Thisview includes the frame 710, the display assembly 720, a display block740, and the eye 155. The display assembly 720 supplies image light tothe eye 155. The display assembly 720 houses the display block 740. Forpurposes of illustration, FIG. 7B shows the cross section 730 associatedwith a single display block 740 and a single eye 155, but in alternativeembodiments not shown, another display block which is separate from thedisplay block 730 shown in FIG. 7B, provides image light to another eyeof the user.

The display block 740, as illustrated below in FIG. 7B, is configured tocombine light from a local area with light from computer generated imageto form an augmented scene. The display block 740 is also configured toprovide the augmented scene to the eyebox 150 corresponding to alocation of a user's eye 155. The eyebox 150 is a region of space thatwould contain a user's eye while the user is wearing the HMD 700. Thedisplay block 740 may include, e.g., a waveguide display, a focusingassembly, a compensation assembly, or some combination thereof. Thedisplay block 740 is an embodiment of the display blocks discussed abovewith regard to FIGS. 1-3. Additionally, the display block 720 includesmay include an immersed Fresnel assembly as discussed above with regardto FIGS. 4-6.

The HMD 700 may include one or more other optical elements between thedisplay block 740 and the eye 155. The optical elements may act to,e.g., correct aberrations in image light emitted from the display block740, magnify image light emitted from the display block 740, some otheroptical adjustment of image light emitted from the display block 740, orsome combination thereof. The example for optical elements may includean aperture, a Fresnel lens, a convex lens, a concave lens, a filter, agrating, a waveguide element, or any other suitable optical element thataffects image light. The display block 740 may be composed of one ormore materials (e.g., plastic, glass, etc.) with one or more refractiveindices that effectively minimize the weight and widen a field of viewof the HMD 700.

FIG. 8A is an example array 800 of sub-pixels on the display element110. The example array 800 shown in FIG. 8A includes red sub-pixels 810,blue sub-pixels 820, and green sub-pixels 830. For example, the array800 is portion of a PENTILE® display. In other embodiments, the array800 may be in some other configuration (e.g., RGB).

A dark space 840 separates each sub-pixel from one or more adjacentsub-pixels. The dark space 840 is a portion of the array 800 that doesnot emit light, and may become visible to a user under certaincircumstances (e.g., magnification) causing the screen door effect thatdegrades image quality. As discussed above in conjunction with FIG. 1,the Fresnel assembly 120 can serve to reduce fixed pattern noise so thedark space 840 between the sub-pixels is not visible to the user (e.g.,by blurring each sub-pixel, creating a blur spot associated with eachsub-pixel in the image). The blur spots are large enough so adjacentblur spots mask the dark space 840 between adjacent full pixels. Inother words, for any display panel, the largest pixel fill-ratio is100%, if there is no gap at all between sub-pixels. However, tocompletely get rid of the screen door artifact on the panel side, thepixel fill-ratio may be much greater (e.g., 300%), such that thesub-pixels of different colors are overlapping. This way, when onlygreen pixels are emitting light, for example, when viewed with perfectviewing optics, there would be no gap between the sub-pixels. This isdifficult to do for OLED and/or LCD display panels, but it is doablewith a diffractive element such as the prism array redirection structure125.

FIG. 8B is an example illustrating adjustment of image data of the array800 of FIG. 8A by the Fresnel assembly 120. As shown in FIG. 8B, each ofthe sub-pixels has an associated blur spot. Specifically, the redsub-pixels 810 have a corresponding red blur spot 860, the bluesub-pixels 820 have a corresponding blue blur spot 870, and the greensub-pixels 830 have a corresponding green blur spot 880. A blur spot isan area filled with an image of a blurred sub-pixel. So long as a blurspot does not overlap with a point of maximum intensity of an adjacentblur spot that is created by a sub-pixel of the same color, the two blurspots are resolvable as two adjacent pixels. In some embodiments, thethree sub-pixels all overlap and creates a white pixel. The shape of theblur spot is not necessarily a circle, but is rather an area includingthe blurred image of a sub-pixel. The redirection structure 125 of FIG.1A can be configured to blur each sub-pixel so the blur spots mask thedark space 940 between adjacent pixels.

ADDITIONAL CONFIGURATION INFORMATION

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the disclosure be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

1. A head-mounted display (HMD) comprising: a display element configuredto output image light in a first band of light through a displaysurface; an optics block configured to direct light from the displayelement to an eyebox, and the eyebox is a region in space occupied by aneye of the user; a Fresnel assembly configured to transmit light in thefirst band and direct light in a second band different than the firstband to a first position; an eye tracking system configured to generateeye tracking information, the eye tracking system comprising: anillumination source configured to illuminate portions of the eyebox withlight in the second band, a camera located in the first position, thecamera configured to capture light in the second band corresponding tolight reflected from the eye of the user through the optics block andreflected by the Fresnel assembly, and a controller configured todetermine eye tracking information using the captured light in thesecond band, and wherein the HMD is configured to adjust the image lightbased on the eye tracking information.
 2. The HMD of claim 1, whereinthe display element is configured to output image light via a pluralityof sub-pixels, sub-pixels separated from each other by a dark space, andthe Fresnel assembly is configured to generate blur spots in the imagelight that mask the dark space between adjacent sub-pixels, with eachblur spot corresponding to a blurred image of a sub-pixel in the imagelight.
 3. The HMD of claim 1, wherein the light in the first band is ina visible spectrum and the light in the second band is in an infraredspectrum.
 4. The HMD of claim 1, wherein the light in the first band isin one polarization and the light in the second band is in an orthogonalpolarization.
 5. The HMD of claim 1, wherein an immersion layer iscoupled to the Fresnel assembly, wherein the immersion layer has thesame refractive index as that of a material from which the Fresnelassembly is composed.
 6. The HMD of claim 5, wherein the Fresnelassembly is directly coupled to the display surface of the displayelement.
 7. The HMD of claim 5, wherein the Fresnel assembly includes aplurality of reflective surfaces, and light incident within a firstangular range at the immersion layer is redirected to the first positionafter at most two reflections off of some of the plurality of reflectivesurfaces.
 8. The HMD of claim 7, wherein light incident within a secondangular range at the immersion layer is redirected to the first positionafter at most two reflections off of some of the plurality of reflectivesurfaces and undergoing at most a single total internal reflection offof an immersion layer-air interface, and the first angular range isdifferent than the second angular range.
 9. The HMD of claim 1, whereinthe Fresnel assembly includes a first set of reflective surfaces and asecond set of reflective surfaces, and the HMD further comprises: asecond camera at a second position, and light in the second bandincident on the first set of reflective surfaces is reflected toward thefirst position, and light in the second band incident on the second setof reflective surfaces is reflected toward the second position.
 10. TheHMD of claim 1, wherein the Fresnel assembly includes a first set ofreflective surfaces and a second set of reflective surfaces and theillumination source is positioned at the second position such that lightin the second band emitted by the illumination source is reflected bythe second set of reflective surfaces toward the eyebox, and light inthe second band incident on the first set of reflective surfaces isreflected toward the first position.
 11. The HMD of claim 1, wherein theFresnel assembly includes a plurality of reflective surfaces that arecoated with a dichroic film that is reflective in the second band oflight and transmissive in the first band of light.
 12. The HMD of claim1, wherein a distance along an optical axis between the Fresnel assemblyand the optics block is closer to the optics block than a distance fromthe optics block to the display element, and the Fresnel assembly ispositioned at a first angle to the optical axis such that it is cantedto direct light to the first position.
 13. A head-mounted display (HMD)comprising: a display element configured to output image light in afirst band of light through a display surface; an optics blockconfigured to direct light from the display element to an eyebox, andthe eyebox is a region in space occupied by an eye of the user; aFresnel assembly configured to transmit light in the first band anddirect light in a second band different than the first band to a firstposition; an illumination source configured to illuminate portions ofthe eyebox with light in the second band; and a camera located in thefirst position, the camera configured to capture light in the secondband corresponding to light reflected from the eye of the user andreflected by the Fresnel assembly.
 14. The HMD of claim 13, wherein adistance between the optics block and the display element is less than adistance between the Fresnel assembly and the display element.
 15. TheHMD of claim 13, wherein an immersion layer is coupled to the Fresnelassembly, wherein the immersion layer has the same refractive index asthat of a material from which the Fresnel assembly is composed.
 16. TheHMD of claim 15, wherein the Fresnel assembly is directly coupled to thedisplay surface of the display element.
 17. The HMD of claim 15, whereinthe Fresnel assembly includes a plurality of reflective surfaces, andlight incident within a first angular range at the immersion layer isredirected to the first position after at most two reflections off ofsome of the plurality of reflective surfaces.
 18. The HMD of claim 17,wherein light incident within a second angular range at the immersionlayer is redirected to the first position after at most two reflectionsoff of some of the plurality of reflective surfaces and undergoing atmost a single total internal reflection off of an immersion layer-airinterface, and the first angular range is different than the secondangular range.
 19. The HMD of claim 12, wherein the Fresnel assemblyincludes a first set of reflective surfaces and a second set ofreflective surfaces, and the HMD further comprises: a second camera at asecond position, and light in the second band incident on the first setof reflective surfaces is reflected toward the first position, and lightin the second band incident on the second set of reflective surfaces isreflected toward the second position.
 20. The HMD of claim 12, whereinthe Fresnel assembly includes a first set of reflective surfaces and asecond set of reflective surfaces and the illumination source ispositioned at the second position such that light in the second bandemitted by the illumination source is reflected by the second set ofreflective surfaces toward the eye, and light in the second bandincident on the first set of reflective surfaces is reflected toward thefirst position.